Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[basicdataset] Add PennTreebank dataset #1580

Merged
merged 6 commits into from
May 3, 2022
Merged

[basicdataset] Add PennTreebank dataset #1580

merged 6 commits into from
May 3, 2022

Conversation

AKAGIwyf
Copy link
Contributor

Description

Add a version of Penn Treebank which is free on github but without POS tags as Torchtext, it had been pre-processed.

close #1579.

zachgk
zachgk reviewed Apr 29, 2022
View reviewed changes
Copy link
Contributor

@zachgk zachgk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes. I made a small change to fix the license. If you haven't done much with licenses, the Apache2 license is typically used for code but there are different licenses used for other kinds of content (books, movies, documents, papers, datasets, artwork, etc). It is important to get the license right and follow the terms because the license describes what you are legally allowed to do with the content.

Besides that, the metadata is uploaded, so you can not modify your tests to use the DJL central repository instead of the local repository.

basicdataset/src/test/java/ai/djl/basicdataset/PennTreebankTextTest.java Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Apr 29, 2022 •
edited
Loading

Codecov Report

Merging #1580 (8ece361) into master (bb5073f) will decrease coverage by 1.21%.
The diff coverage is 53.04%.

@@             Coverage Diff              @@
##             master    #1580      +/-   ##
============================================
- Coverage     72.08%   70.87%   -1.22%     
- Complexity     5126     5427     +301     
============================================
  Files           473      507      +34     
  Lines         21970    23757    +1787     
  Branches       2351     2587     +236     
============================================
+ Hits          15838    16837     +999     
- Misses         4925     5630     +705     
- Partials       1207     1290      +83
Impacted Files Coverage Δ
api/src/main/java/ai/djl/modality/cv/Image.java 69.23% <ø> (-4.11%) ⬇️
...i/djl/modality/cv/translator/BigGANTranslator.java 21.42% <ø> (-5.24%) ⬇️
...odality/cv/translator/BigGANTranslatorFactory.java 33.33% <0.00%> (+8.33%) ⬆️
...nslator/InstanceSegmentationTranslatorFactory.java 14.28% <0.00%> (-3.90%) ⬇️
.../modality/cv/translator/YoloTranslatorFactory.java 8.33% <0.00%> (-1.67%) ⬇️
...i/djl/modality/cv/translator/YoloV5Translator.java 5.69% <0.00%> (ø)
...odality/cv/translator/YoloV5TranslatorFactory.java 8.33% <0.00%> (-1.67%) ⬇️
...pi/src/main/java/ai/djl/ndarray/BytesSupplier.java 54.54% <0.00%> (-12.13%) ⬇️
...pi/src/main/java/ai/djl/repository/Repository.java 83.33% <ø> (ø)
...l/training/loss/SigmoidBinaryCrossEntropyLoss.java 64.00% <0.00%> (ø)
... and 243 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1d62d6d...8ece361. Read the comment docs.

@zachgk zachgk merged commit 0b5fee8 into deepjavalibrary:master May 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zachgk zachgk zachgk approved these changes

@frankfliu frankfliu Awaiting requested review from frankfliu

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add Penn Tree Bank Dataset
3 participants
@AKAGIwyf @codecov-commenter @zachgk